Forward-merge branch-24.10 into branch-24.12

Forward-merge branch-24.10 into branch-24.12

This project currently lists `gcovr` (https://pypi.org/project/gcovr/) as a pip dependency for development. I strongly suspect that that was unintentional... it doesn't look like it has any reliance on getting that package via `pip` (just conda, in the C++ test jobs and for local C++ development).

This proposes removing `gcovr` from the list of pip dependencies, so it won't get installed in the DLFW images or other places where `rapids-make-pip-env` from https://github.com/rapidsai/devcontainers is called.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1688

Rather than hand-coding a fraction of the device memory use the utility routine.

- Closes #1674

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Rong Ou (https://github.com/rongou)

URL: #1689

…hon classes public (#1676)

Closes #1280

Authors:
  - Matthew Murray (https://github.com/Matt711)
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1676

This PR fixes a bug in #1676. It makes sure that rmm imports work correctly using both `from rmm._lib...` and `import rmm._lib...` syntax.

I'm adding DO NOT MERGE until I do some more testing.

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1693

Contributes to rapidsai/build-planning#94

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #1695

Forward-merge branch-24.10 into branch-24.12

NVIDIA/cuda-python#137 reorganised the low-level binding structure which broke our cross-linking, update to the new name to fix.

- Closes #1698

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Matthew Murray (https://github.com/Matt711)
  - Bradley Dice (https://github.com/bdice)

URL: #1699

This means that downstream libraries that get their `fmt` dependency from RMM will use `-DFMT_SHARED` in the compile command: this matches what the rapids combined devcontainers do, so we get sccache hits.

- Closes #1701

Authors:
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

URL: #1702

Contributes to rapidsai/build-planning#106

Proposes specifying the RAPIDS version in `conda install` calls in CI that install CI artifacts, to reduce the risk of CI jobs picking up artifacts from other releases.

Authors:
  - James Lamb (https://github.com/jameslamb)
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: #1696

Small fix to some typos that cropped up in the .gitignore with #1676

Authors:
  - Charles Blackmon-Luca (https://github.com/charlesbluca)
  - Lawrence Mitchell (https://github.com/wence-)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Mark Harris (https://github.com/harrism)

URL: #1697

Follow-up to #1696

Changes relative to that PR:

* switches to pinning CI conda installs to the output of `rapids-version` (`{major}.{minor}.{patch}`) instead of `rapids-version-major-minor` (`{major}.{minor}`), to get a bit more protection in the presence of hotfix releases
* restores some exporting of variables needed for docs builds

In #1696, I'd missed that this project's Doxygen setup is expecting to find `RAPIDS_VERSION_MAJOR_MINOR` defined in the environment.

https://github.com/rapidsai/rmm/blob/90a5631e1093ce44c4feceb88fcf557c3dfc043b/ci/build_docs.sh#L36

https://github.com/rapidsai/rmm/blob/90a5631e1093ce44c4feceb88fcf557c3dfc043b/doxygen/Doxyfile#L41

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Mike Sarahan (https://github.com/msarahan)

URL: #1703

Contributes to rapidsai/build-planning#104




This PR removes support for accessing rmm's underlying spdlog logger directly.

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Lawrence Mitchell (https://github.com/wence-)
  - Mark Harris (https://github.com/harrism)

URL: #1690

Closes #1705.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #1706

Fixes issue brought up in rapidsai/rapids-cmake#634 (comment) where rmm wasn't using rapids_cmake_support_conda_env

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)
  - Mark Harris (https://github.com/harrism)

URL: #1707

This PR is replacing the `VAULT_HOST` variable with `AWS_ROLE_ARN`. This is required to use the new token service to get AWS credentials.

Authors:
  - Jordan Jacobelli (https://github.com/jjacobelli)

Approvers:
  - Paul Taylor (https://github.com/trxcllnt)
  - Bradley Dice (https://github.com/bdice)

URL: #1708

Contributes to rapidsai/build-planning#111

Proposes some small packaging/CI changes, matching similar changes being made across RAPIDS.

* printing `sccache` stats to CI logs
* updating to the latest `rapids-dependency-file-generator` (v1.16.0)
* reducing verbosity of `pip wheel` in wheel builds

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)

URL: #1712

Follows up #1676 to add deprecation warnings to the `rmm._lib` sub package.

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1713

The `is_host_memory` function has been updated to understand that `cudaMemoryTypeUnregistered` is returned when provided pointers allocated by `malloc` and other host side allocation functions.

The `is_device_memory` function has been restricted to report only when device pointer that is usable by the calling cuda context. For that reason the tests now also set the active cuda device for all calling threads.

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Rong Ou (https://github.com/rongou)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1714

Close #830 .

- Add the arena allocator to the public Python interface.
- Small changes to the logger initialization to avoid exposing spdlog in the shared objects.

Authors:
  - Jiaming Yuan (https://github.com/trivialfis)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1711

Due to a bug in cuda-python we must disallow cuda-python 12.6.1 and 11.8.4. See rapidsai/build-planning#116 for more information.

This PR disallows those versions, and other changes following from that:

* specifying `python` in both `host:` and `run:` dependencies for the `rmm` conda package
* ignoring deprecation warnings raised by newer versions of `cuda-python`

Authors:
  - Bradley Dice (https://github.com/bdice)
  - James Lamb (https://github.com/jameslamb)
  - https://github.com/jakirkham

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #1720

Follow-up to #1720

Contributes to rapidsai/build-planning#116

That PR used `!=` requirements to skip a particular version of `cuda-python` that `rmm` was incompatible with. A newer version of `cuda-python` (12.6.2 for CUDA 12, 11.8.5 for CUDA 11) was just released, and it also causes some build issues for RAPIDS libraries: rapidsai/cuvs#445 (comment)

To unblock CI across RAPIDS, this proposes **temporarily** switching to ceilings on `rmm`'s `cuda-python` dependency.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1723

…CI (#1716)

Follow-up to #1613

Similar to rapidsai/cudf#17131

Proposes using the new `rapids-generate-pip-constraints` tool from `gha-tools` to generate a list of pip constraints pinning to the oldest supported verisons of dependencies here.

## Notes for Reviewers

### How I tested this

`wheel-tests`:

* oldest-deps: numpy 1.x ([build link](https://github.com/rapidsai/rmm/actions/runs/11620907528/job/32364032641?pr=1716#step:8:106))
* latest-deps: numpy 2.x ([build link](https://github.com/rapidsai/rmm/actions/runs/11620907528/job/32364032835?pr=1716#step:8:104))

And the testing of the general approach in rapidsai/gha-tools#114 (comment)

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1716

Contributes to rapidsai/build-planning#110

Proposes adding 2 types of validation on wheels in CI, to ensure we continue to produce wheels that are suitable for PyPI.

* checks on wheel size (compressed),
  - *to be sure they're under PyPI limits*
  - *and to prompt discussion on PRs that significantly increase wheel sizes*
* checks on README formatting
  - *to ensure they'll render properly as the PyPI project homepages*
  - *e.g. like how https://github.com/scikit-learn/scikit-learn/blob/main/README.rst becomes https://pypi.org/project/scikit-learn/*

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1726

…replay benchmark (#1728)

Fixes #1727
Contributes to rapidsai/build-planning#26

 - Removes `-Wno-error=deprecated-declarations`
 - Replaces deprecated usage of `rmm::logger()` in reply benchmark with supported `RMM_LOG_INFO` macros.

Note the latter duplicates a change in #1724 which allows the two PRs to be merged independently.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Rong Ou (https://github.com/rongou)
  - Bradley Dice (https://github.com/bdice)

URL: #1728

Fixes #1717
Also fixes #1710 in 5330063

I have replaced fmt-style format string placeholders (`"... {} ..."`) with printf-style placeholders by adding a function `rmm::detail::formatted_log()`, which I modified from @vyasr 's #1722.

~The only remaining mention of fmt is in CMakeLists.txt. Do we still need to explicitly fetch fmt?~
Removed.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1724

close #1691

Authors:
  - Mike Sarahan (https://github.com/msarahan)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1692

…1667)

We can remove the optimizations around  `CUDA_STATIC_RUNTIME` and instead see if the function is already in the process space so that RMM doesn't need to have any build context to run properly

Fixes #1679

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #1667

Fixes #1733 by querying total device memory and using twice as much in tests that are expected to fail allocation.

Authors:
  - Mark Harris (https://github.com/harrism)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #1734